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ABSTEACT 

A iirief paper presented at the American Educational 
Eeaearch Association Annual meeting (April 1977) deacxibea the 
development, and analysis of the Inatructional Management System 
(IMS) which is used in the Lanaing elementary schcois* Ihe report is 
divided into three sectiona: a hiatorical overview of the program, a 
description of IMS, and an aaaessment of the advantages, 
disadvantages, and outcomes as assessed by the teacher* In «^ effort 
to make the lowest achieving students successful, the teaching of 
30,000 elementary students of diverse backgrounds and socioeconomic 
status in 56 schools aas achieved by decentralization of deciaion 
making on a school-^by--school basis, teacher participation, parental 
involvement, and implementation of IMS^ Iheae actions were mandated 
by Elemeritary Secondary Education Act, Title I funds used for the 
program. The implementation of the IMS includes 140 reading 
objectives and 150 math objectives which are representative of skills 
students in all schools should master. Accompanying each objective 
are pre-tests and post-testa* .Additionally, record keeping carda for 
reading and math are utilized to plot pupil progress through the 
ayatem* Advisory committeea are cdiprised of teachers, administrators 
parents, and evaluatora who review items, auggest time-'saving testing 
technigues and effective management practices* Ihe positive and 
negative raaults of t^e development and implementation of IMS are ^ 
reported. The advantages of IMS are- immediate information on skill 
level, appropriate educational objectives, greater test reliability, 
and better communication between teachers and parents* The 
disadvantages enumerated by the teachers arei feelings of being 
pressured to achieve a certain number of objectives, and very little 
input into the development of tl^e IMS components. Additionally, 
students are occasionally overteslied; time reguired to master 
particular Ob jectives is not equal; and collecting, recording, and 
analyzing data is too time consuming* In the IMS program, teacher 
confidence is increased and teacher morale is strengthened as a 
result of greater teacher participation. As a reault of the success 
of this IMS program a system similar to it is being developed for the 
secondary level* (Author/JP) 
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TH?5 OOCU/V^ENT HAS BEEN REPRO- 
OUCED EXACTLY AS RiCeiVlD FROM 
T:^g PERSON OR ORGANIZATION ORiGlN- 
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liTATED DO NOT NECfSSARjLY REPRE- 
^ENTQFPlCiAL NA f lONAL !NSTlTUTB OF 
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This paper descrihes the developinont and implenientatlon of 
a program evaluation model which provides for 1) local teacher 
ov/nersh1p5 2) on-going revision for validity and reliability 5 
3) testing appropriate to teaching^ 4) meaningful achievement 
information for parents^ and 5) one evaluation design for all 
compensatory education programs. The work defcribsd In this 
paper was buITt upon the premise that it is appropnate and 
necessary to "re-Invent the -wheel" when planning and imple- 
menting objectives-based avaluation designs which ssrve both 
Instructionar decision-making and funding source accountabil- 
ity. 
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To put the information in perspective s a deomgraphlc description 
of the school district seems appropriate, Lansing is the capitol 
city of Michigan, located near the center of the lower peninsula. 
The Lansing School District serves some 30,000 students in 56 
school buildings. The student composition is a most diverse one 
by any criterion. In terms of races the Lansing SchooT' District 
enrollment is' comprised of approximately 18% BlackrlOl LatinOs 
Z% Native American and Oriental and 70% Caucasian. Socio-economic 
callys the students range from ADC and welfare (ZOSf to the off- 
spring of professional and upper level managerial families. 



Three efforts in the past five years have been designed to better 
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serve this diverse popuiatlon. First, decentralization of decision- 
makings called Responsible Autonomy, has called for Instructional 
decisions to be made as close as possible to the students,- that Is 
at the bunding level. Finances and personriel are allocated to 
Individual buildings and educational priorities are established with 
input of staff, principal and community* Second, desegregation which 
began as a voluntary effort and became court ordered has resulted 1n 
two-way bussing of students 1n 20 of 46 elementary schools. The 
other elementary school s have been Integrated through neighborhood 
housing patterns and the secondap schools have been Integrated through 
boundary adjustments. The third efforts the development of the 
Instructional Management System (IMS)^ Is the major topic of this paper. 

The Instructional Management System, IMS, was designed to have a timely 
Impact on decision-making at the classroom^ school and district level. 
To do so? it first had to have the support of teachers* Several ap- 
proaches were adopted to achieve this objective. The most important 
was accepting the necessity of "Re^-Inventing the Wheel". The Impor- 
tance of writing "Lansing'* objectives was recognized^ even though these 
might provfe quite similar to others already available. Teacher-developed 
items were chosen over commercial or expert -written items. In each in- 
stance, it was decided to focus on effectiveness rather than efficiency* 
That is, it w.as_more important that an IMS have credibility in the 
schools, than it was to develop the matertal efficiently* , 

A second aspect of these efforts was the necessity for the evaluation 
design to support the district policy of responsible autonomy f 1 *e* ^ to 
move Instructional decision-making to the building Tevel. This meant 



that the IMS would have to have impact on what happens dany'^in the 
classroom* Ftirther, it meant that the system would have to respects 
indeed encourage, diverse approaches to teaching- 
Third, the obligations imposed and the funds granted by the federal 
and state conipinsatory education programs (Title T and Chapter 3) were 
used as levers to achieve these changes in the district as a whole* 
When possible J demands imposed by these programs were viewed as oppor- 
tunities to improve the district's capacity to serve students. It 
was essential that the IMS be acceptable to outside funding agencies. 

This paper consists of r-^an historical sketch of the development of 
the IMSs a description of the IMS as it operates now, and finally an 
analysis of the advantages and disadvantages of the approach taken 
with a preview of next steps. - 

" — A m%tory, 1971-1975 

During the 1971-72 school year^ each school building was asked to 
select one district goal* set a behavipraT objective related to the 
goal j develop a pre-testj instructional units and a post-test for 
this objective and, finallyt carry out this teirGhs test and recyclei.. 
if necessary, model. The approach was low key. ' The choice of, content 
was wide openy the number of students and staff involved was decided at 
the building level. The evaluation unit focused 'its efforts on working 
with those buildings which indicated an interest* This. first step 
resulted in seyeraT positive outcomes. Awareness of instructional objec 
, tives and the test/teach/test_rTOdel Increased* .Staff* enthusiastic and: 
capable in these aspects^ were identified and recognized, A first - ■ 
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Handbook of Objectives , was produced, Evaluators and teachers were 
working together. 

The same year all Lansing elementary .schools became Invol ved in 
the state-funded compensatory education program. Chapter 3, For many 
staffs this was the first contact with special programs. It meant more ^ 
moneyp but more testing. It meant trying to make the lowest achieving 
students successful. Most controversially, 1t threatened to penalize 
schools that did not perform well on norm-referenced tests. There was 
considerable support among the teaching staff for finding an evaluation 
design which was not based on , norm-referenced gain scores. Such a 
design would have to provide feedback before the end of the year* It, 
toos would have'to take Into account the range of achievement within 
each grade, 

In the following years several opportunltfes were found for developing 
an objectives -based evaluation design. During 1972-73, criterion re- 
ferenced tests were mandated by the State Department for evaluating 
the kindergarten and first grades of Chapter 3. These objectives arid 
test Items were developed by Lansing teacher committees and foreshadowed 
further efforts. The Office of Eval uatlon Services provided leaderships 
technical and (very Important) clerical support to these efforts, 
During this time, also, several pilots of commercial objective referenced 
testing systems were made. ' . . 

During the scfiool years 72-73 and 73-74, the Middle Cities Consortlaj 
of which Lansing was a member, was awarded Title III grant to develop 
instructional management systems, including objectives, tests, teaching 



prescriptions I and inservice packages. Housed .in a Lansing elementary 
schools this project provided considerable opportunity for teachers to 
work on developing this system. 

As the result of extensive -:d1scuss1on with the Michigan Department of 
Educations Lansing was able to plot student progress in the "Middle ' 
Cities" system as the Title I evaluation for 1974-75. Building on what 
had been developed In the Middle' Cities model and what had been devel- 
oped internally in the Lansing School District* the Instructtonal 
Management System was packaged and disseminated to all Lansing schools 
for implementation during 75-76* 

1 A Description of the IMS 

And Further Developments 

The Instructional Management System 1n Its present formt has three (3) 
parts. It 'should be pointed out that this system Is neither final nor 
Ideal. It Is simply where we are now. First there are 140 reading and 
150 math objectives which have been identified as "critical"* These 
are considered to represent skills which students in all school buildings 
should master* Additional objectives may^ and usually are^ set fay a 
particular teacher or school. The objectives are not tied to grade 
levels* but rather are considered to be hierarchical; students may be 
working an^^here along the continuum. Accofnpanylng each objective are a 
pre-test and a post-test* Finally, record keeping cards, one for math 
and one for reading, are used to plot pupil progress through the system. 
Instructional activities were not provided in keeping with tht plan for 
each building to determine the best way to provide for instruction. 
Progress of all elementary students, parti cularly those in specially 
funded- programs, is plotted against the IMS objectives. The objectives 



and tests, theni are comnxin to all buldings. At the same time, sup- 
plemental objectives and cufricula are the responsibility of individual 
buildings. 

Once the implementation of the IMS became a district -wide requirement, 
teacher resistance occurred. Teachers complained of not being involved 
in either the development of the system or in the decision to implement 
it district --wide. They felt unprepared to participate on such a large 
scale and questioned the quality of the objectives and items. It be- 
came clear that if the IMS was going to be given a chance to succeed, 
teacher involvement and teacher in-service had to occur on a larger , 
more systematic scale. A first step was the establishment of the Joint 
Committee on the IMS. 

This coirimittee was comprised of teachers (one per grade level )p ad- 
ministrators and parents. A number of recommendations grew out of the 
committee* The recormiendations focused on four basic areas: 1) the im- 
provement of reading test items, 2) the incorporation of teacher Judg- 
ment as a means of assessment for kindergarten and first grade objec'- 
tives, 3) the reduction of the pre-testing in math as much as was posslbl 
and 4) the Inservicing of all teachers at the beginning of 1976-77 to 
familiarize them with changes and the requisite management skills to im- 
plement the system. Each of these recommendations^ was adopted and im- 
plemented as suggested. 

During this school year, 76-77, another avenue for soTiciting teacher in- 
put has been the use of Grade Level Steering Committees, These are 
groups of teachers at specific grade levels who review items, suggest 



time-saving testing and effective management practices. This system 
of teacher input is helping to clarify and correct some of the pro- 
blems of implementation. Evaluation staff and administrators coordinate 
these activities in the sense that they help bring a group together and 
provide technical assistance and financing for these groups. 

Establishing the validity and reliability of the objectives and tests 
was Important for two reasons: teachers wanted assurance that the 
system they were being asked to implement was a quality one and the State 
Department of Education wanted similar assurance that the tests and ob- 
jectives of the IMS were psychometrically sound* Content validity data 
were gathered through several methods. The near-daily interchange of ^ 
communication between the evaluators and the teachers provided informal 
feedback on items* Each elementary teacher was formally invited to 
submit suggestions for revision in writing to the Department of Elementary 
Education. ^Further suggestions came through the Grade Level Steering 
Committees and from subject matter specialists* The Program for Equal 
OppoHunity (PEO) staff at the University of Michigan reviewed all items 
for racial* sexual and ethnic bias. 

Reliability co-efficients were computed by evaluation staff using the 
Kuder-Rlchardson 20 formula* Tests with a reliability index of less 
than were revised* For reading that was 71 of the tests and for 
math' 5% of the tests* ^ = ... 

Once the validity and reliability information was gathered teachers 
rewrote the items determined to be either invalid and/or unrel iable. 
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It is assumed that further revisions will be made at tha end of this 
year based on the work of teacher committees and additional reliability 
checks . 

An Analysis 

The following and final section of this paper deals with the positive 
and negative results of the development and use of the IMS in Lansing* 
This paper will close with: 

Positive Outcomes 

L Teachers are discussing instruction* In their criticisms of 
the system, they addressed' such sophisticated Issues as whether 
one objective is more difficult to achieve than another^ 
whether or not four items are enough to assure knowledge, 

2, As students move from one school to another and possibly from 
-one reading system to another? the receiving teacher has Imnedlate 

information on skill level* This is seen as especially Impor- 
tant in an urban system where both the intra-district and inter- 
district mobility rate is high- 

3. The district has a program evaluation model which reflects on- 
going classroom instruction and which Is approved by major 
funding sources. Given the fact that there are some 5600 students 
eligible for sped ally funded programs and that 43% of those 
students are in two or more such programs ^ having one testing 
model which Interfaces with the on-going curriculum Is seen as 
beneficial to both students and teachers. 

4- The data show the tests to be surprisingly reliable given their 
short length. Reliability was estimated at ,75 for reading and 



,82 for mathematics. * These data helped assure teachers and 
funding officials of the quality of the items* 

5* Parents receive additional Information on their children's 
educational needs during parent-teacher conferences. Teachers 
report that having record cards showing pupil progress on basic 
skills 1s helpful in facilitating communication* 

6. Emphasis has been placed on teaching skills based on identified 
students needs, not on what comes next in the book. 

Negative Outcomes^ 

1, Many teachers feel that they are unprepared for implenientlng 
the system or that they have had no Input Into the development 
of the components. They still have concerns about the quality 
of the system. 

2, Many teachers are not Implementing the Instructional Managemeht 
System as It was originally designed* There are cases of 
massive pre-testing or massive post=test1ng thereby diminishing 
the usefulness of results and causing overtesting of students. 

3* Teachers report they feel pressured to "achieve" a certain 

number of objectives especially for compensatory program students 

4, firing the second year of implementation teachers have com- 
plained that the previous year*s results are inaccurate and 
that the students have not necessarily mastered the skills in- 
dicated. 

5. The objectives are not "equar- In the sense that the length 
of time required to master a particular objective can vary 
considerably. 
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6* While there locator placement tests in math, these 
kinds of tests are not a^anable 1n reading. There is a high 
likelihood that sinniap tests will be developed for the reading 
Component* 

7, Recording data is a terribly time cGnsuming activity for 
teachers. Collecting and analyzing the data for program, 
evaluation purposes, also time consuming and costly* 



With this the second yea^ of IMS Implementation » much of the initial 
hostility towards the systern has subsided* Teachers seem reassured by 
the actions of the joint Committee and Grade Level Committees, This 
does not mean ey^^yonB likes the IMS or agrees on all facets of It, 
but there is consensus that studs^t progress will be assessed during 
the yesr and that the system will not simply "go away". Changes made 
as a result of committee ^ork have strengthened teacher ov/nership 
because teachers c^n see that the1^ input 1s valued and acted upon. 
This committee pJ^ocess ls being continued as further changes 1n the 
IMS are made. A Systern similar tci and based on the elementary IMS 
1s being developed at the Second^^^y levels. Using the title Life 
Role Competencies rather than critical objectives^ It is planned that 
this system will be impleniented over a three year period. This effort 
win bs developed locally by Lansing teachers for Lansing students. 
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